Focused Reading: Reinforcement Learning for What Documents to Read

Recent efforts in bioinformatics have achieved tremendous progress in the machine reading of biomedical literature and the assembly of the extracted biochemical interactions into largescale models such as protein signaling pathways. However, batch machine reading of literature at today’s scale (PubMed alone indexes over 1 million papers per year) is infeasible due to both cost and processing overhead. In this work we propose focused reading as an alternative. Focused reading casts machine reading as a search process where queries are defined by pairs of entities, e.g., proteins, to be connected through the models extracted from the literature. Our approach casts the task as an incremental search over the graph of biochemical interactions, where each focused reading step makes a choice between widening the search space (exploration), or focusing on the most relevant documents (exploitation). We learn strategies that distinguish between exploration and exploitation using reinforcement learning (RL), and demonstrate that an RL-learned strategy is capable of answering more queries while reading fewer papers than a strong deterministic baseline.